Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a basic E2E test suite #638

Merged
merged 2 commits into from
Jan 28, 2025
Merged

Implement a basic E2E test suite #638

merged 2 commits into from
Jan 28, 2025

Conversation

hanzi
Copy link
Collaborator

@hanzi hanzi commented Jan 23, 2025

Description

This adds a test suite based on the unittest module (included in Python by default) that runs some end-to-end tests.

The basic principle is that for each test it will instantiate the emulator with a given save state, run some bot code on it (as if you were writing a bot mode) and assert whatever needs to be asserted.

I have already added mocks for the ask_for_choice() function (there is a set_next_choice("selection") utility function to configure what it should return) and the wait_for_unique_rng_value() helper. The latter can be configured using set_next_rng_seed(0x1234abcd), which will also immediately write that RNG value to memory. I've used this to create test cases that are guaranteed to enocunter shiny Pokémon to test handling for that.

Tests can be run by calling python -m unittest discover -s tests from the repository root. This is also described in a Readme file in the tests/ directory.

Coverage

For now, I've added test cases for some pathfinding features, some battle handling features, the Spin mode and the Starters mode. More will be added in other PRs in the future.

I'm only testing English versions of Emerald, Ruby, and FireRed. I decided against testing Sapphire and LeafGreen just to make things a bit simpler and to improve the runtime of the test suite -- mechanically, these games are so similar to Ruby and FireRed that I don't think we have to test against those too.

Changes to the bot itself

Apart from the test suite (in the tests/ directory), I had to make some changes:

  • context has a new flag that indicates the bot is in testing mode. I want to try and use that as little as possible to we don't pollute the bot code with lots of special branches for testing.
  • For now, this is only used by the LibmgbaEmulator class and the handle_encounter() function to disable some features we really don't want, such as creation of a save state on shutdown and when encountering a shiny/CCF match, some console output, and notably the loading of save games (i.e. *.sav files.)
  • Some map-related code (including pathfinding functions) use global variables to cache their data, but this leads to issues because the test suite will test different games in the same process. So I've updated these caches to use a per-game key.

Checklist

  • Black Linter has been ran, using --line-length 120 argument
  • Wiki has been updated (if relevant)

@hanzi hanzi force-pushed the e2e-tests branch 4 times, most recently from 7758ef4 to 027fd82 Compare January 24, 2025 18:02
@hanzi hanzi merged commit e387c10 into 40Cakes:main Jan 28, 2025
1 check passed
@hanzi hanzi deleted the e2e-tests branch January 28, 2025 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants